A realistic evaluation : the case of protocol-based care

Background 'Protocol based care' was envisioned by policy makers as a mechanism for delivering on the service improvement agenda in England. Realistic evaluation is an increasingly popular approach, but few published examples exist, particularly in implementation research. To fill this gap, within this paper we describe the application of a realistic evaluation approach to the study of protocol-based care, whilst sharing findings of relevance about standardising care through the use of protocols, guidelines, and pathways. Methods Situated between positivism and relativism, realistic evaluation is concerned with the identification of underlying causal mechanisms, how they work, and under what conditions. Fundamentally it focuses attention on finding out what works, for whom, how, and in what circumstances. Results In this research, we were interested in understanding the relationships between the type and nature of particular approaches to protocol-based care (mechanisms), within different clinical settings (context), and what impacts this resulted in (outcomes). An evidence review using the principles of realist synthesis resulted in a number of propositions, i.e., context, mechanism, and outcome threads (CMOs). These propositions were then 'tested' through multiple case studies, using multiple methods including non-participant observation, interviews, and document analysis through an iterative analysis process. The initial propositions (conjectured CMOs) only partially corresponded to the findings that emerged during analysis. From the iterative analysis process of scrutinising mechanisms, context, and outcomes we were able to draw out some theoretically generalisable features about what works, for whom, how, and what circumstances in relation to the use of standardised care approaches (refined CMOs). Conclusions As one of the first studies to apply realistic evaluation in implementation research, it was a good fit, particularly given the growing emphasis on understanding how context influences evidence-based practice. The strengths and limitations of the approach are considered, including how to operationalise it and some of the challenges. This approach provided a useful interpretive framework with which to make sense of the multiple factors that were simultaneously at play and being observed through various data sources, and for developing explanatory theory about using standardised care approaches in practice

The implausibility of ‘usual care’ in an open system: sedation and weaning practices in Paediatric Intensive Care Units (PICUs) in the United Kingdom (UK)

Background: The power of the randomised controlled trial depends upon its capacity to operate in a closed system whereby the intervention is the only causal force acting upon the experimental group and absent in the control group, permitting a valid assessment of intervention efficacy. Conversely, clinical arenas are open systems where factors relating to context, resources, interpretation and actions of individuals will affect implementation and effectiveness of interventions. Consequently, the comparator (usual care) can be difficult to define and variable in multi-centre trials. Hence outcomes cannot be understood without considering usual care and factors that may affect implementation and impact on the intervention. Methods: Using a fieldwork approach, we describe PICU context, ‘usual’ practice in sedation and weaning from mechanical ventilation, and factors affecting implementation prior to designing a trial involving a sedation and ventilation weaning intervention. We collected data from 23 UK PICUs between June and November 2014 using observation, individual and multi-disciplinary group interviews with staff. Results: Pain and sedation practices were broadly similar in terms of drug usage and assessment tools. Sedation protocols linking assessment to appropriate titration of sedatives and sedation holds were rarely used (9 % and 4 % of PICUs respectively). Ventilator weaning was primarily a medical-led process with 39 % of PICUs engaging senior nurses in the process: weaning protocols were rarely used (9 % of PICUs). Weaning methods were variably based on clinician preference. No formal criteria or use of spontaneous breathing trials were used to test weaning readiness. Seventeen PICUs (74 %) had prior engagement in multi-centre trials, but limited research nurse availability. Barriers to previous trial implementation were intervention complexity, lack of belief in the evidence and inadequate training. Facilitating factors were senior staff buy-in and dedicated research nurse provision. Conclusions: We examined and identified contextual and organisational factors that may impact on the implementation of our intervention. We found usual practice relating to sedation, analgesia and ventilator weaning broadly similar, yet distinctively different from our proposed intervention, providing assurance in our ability to evaluate intervention effects. The data will enable us to develop an implementation plan; considering these factors we can more fully understand their impact on study outcomes

Improving quality of care through routine, successful implementation of evidence-based practice at the bedside: an organizational case study protocol using the Pettigrew and Whipp model of strategic change

BACKGROUND: Evidence-based practice (EBP) is an expected approach to improving the quality of patient care and service delivery in health care systems internationally that is yet to be realized. Given the current evidence-practice gap, numerous authors describe barriers to achieving EBP. One recurrently identified barrier is the setting or context of practice, which is likewise cited as a potential part of the solution to the gap. The purpose of this study is to identify key contextual elements and related strategic processes in organizations that find and use evidence at multiple levels, in an ongoing, integrated fashion, in contrast to those that do not. METHODS: The core theoretical framework for this multi-method explanatory case study is Pettigrew and Whipp's Content, Context, and Process model of strategic change. This framework focuses data collection on three entities: the Why of strategic change, the What of strategic change, and the How of strategic change, in this case related to implementation and normalization of EBP. The data collection plan, designed to capture relevant organizational context and related outcomes, focuses on eight interrelated factors said to characterize a receptive context. Selective, purposive sampling will provide contrasting results between two cases (departments of nursing) and three embedded units in each. Data collection methods will include quantitative tools (e.g., regarding culture) and qualitative approaches including focus groups, interviews, and documents review (e.g., regarding integration and “success”) relevant to the EBP initiative. DISCUSSION: This study should provide information regarding contextual elements and related strategic processes key to successful implementation and sustainability of EBP, specifically in terms of a pervasive pattern in an acute care hospital-based health care setting. Additionally, this study will identify key contextual elements that differentiate successful implementation and sustainability of EBP efforts, both within varying levels of a hospital-based clinical setting and across similar hospital settings interested in EBP

Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science

Abstract Background Many interventions found to be effective in health services research studies fail to translate into meaningful patient care outcomes across multiple contexts. Health services researchers recognize the need to evaluate not only summative outcomes but also formative outcomes to assess the extent to which implementation is effective in a specific setting, prolongs sustainability, and promotes dissemination into other settings. Many implementation theories have been published to help promote effective implementation. However, they overlap considerably in the constructs included in individual theories, and a comparison of theories reveals that each is missing important constructs included in other theories. In addition, terminology and definitions are not consistent across theories. We describe the Consolidated Framework For Implementation Research (CFIR) that offers an overarching typology to promote implementation theory development and verification about what works where and why across multiple contexts. Methods We used a snowball sampling approach to identify published theories that were evaluated to identify constructs based on strength of conceptual or empirical support for influence on implementation, consistency in definitions, alignment with our own findings, and potential for measurement. We combined constructs across published theories that had different labels but were redundant or overlapping in definition, and we parsed apart constructs that conflated underlying concepts. Results The CFIR is composed of five major domains: intervention characteristics, outer setting, inner setting, characteristics of the individuals involved, and the process of implementation. Eight constructs were identified related to the intervention (e.g., evidence strength and quality), four constructs were identified related to outer setting (e.g., patient needs and resources), 12 constructs were identified related to inner setting (e.g., culture, leadership engagement), five constructs were identified related to individual characteristics, and eight constructs were identified related to process (e.g., plan, evaluate, and reflect). We present explicit definitions for each construct. Conclusion The CFIR provides a pragmatic structure for approaching complex, interacting, multi-level, and transient states of constructs in the real world by embracing, consolidating, and unifying key constructs from published implementation theories. It can be used to guide formative evaluations and build the implementation knowledge base across multiple studies and settings.http://deepblue.lib.umich.edu/bitstream/2027.42/78272/1/1748-5908-4-50.xmlhttp://deepblue.lib.umich.edu/bitstream/2027.42/78272/2/1748-5908-4-50-S1.PDFhttp://deepblue.lib.umich.edu/bitstream/2027.42/78272/3/1748-5908-4-50-S3.PDFhttp://deepblue.lib.umich.edu/bitstream/2027.42/78272/4/1748-5908-4-50-S4.PDFhttp://deepblue.lib.umich.edu/bitstream/2027.42/78272/5/1748-5908-4-50.pdfhttp://deepblue.lib.umich.edu/bitstream/2027.42/78272/6/1748-5908-4-50-S2.PDFPeer Reviewe

Employing external facilitation to implement cognitive behavioral therapy in VA clinics: a pilot study

<p>Abstract</p> <p>Background</p> <p>Although for more than a decade healthcare systems have attempted to provide evidence-based mental health treatments, the availability and use of psychotherapies remains low. A significant need exists to identify simple but effective implementation strategies to adopt complex practices within complex systems of care. Emerging evidence suggests that facilitation may be an effective integrative implementation strategy for adoption of complex practices. The current pilot examined the use of external facilitation for adoption of cognitive behavioral therapy (CBT) in 20 Department of Veteran Affairs (VA) clinics.</p> <p>Methods</p> <p>The 20 clinics were paired on facility characteristics, and 23 clinicians from these were trained in CBT. A clinic in each pair was randomly selected to receive external facilitation. Quantitative methods were used to examine the extent of CBT implementation in 10 clinics that received external facilitation compared with 10 clinics that did not, and to better understand the relationship between individual providers' characteristics and attitudes and their CBT use. Costs of external facilitation were assessed by tracking the time spent by the facilitator and therapists in activities related to implementing CBT. Qualitative methods were used to explore contextual and other factors thought to influence implementation.</p> <p>Results</p> <p>Examination of change scores showed that facilitated therapists averaged an increase of 19% [95% CI: (2, 36)] in self-reported CBT use from baseline, while control therapists averaged a 4% [95% CI: (-14, 21)] increase. Therapists in the facilitated condition who were not providing CBT at baseline showed the greatest increase (35%) compared to a control therapist who was not providing CBT at baseline (10%) or to therapists in either condition who were providing CBT at baseline (average 3%). Increased CBT use was unrelated to prior CBT training. Barriers to CBT implementation were therapists' lack of control over their clinic schedule and poor communication with clinical leaders.</p> <p>Conclusions</p> <p>These findings suggest that facilitation may help clinicians make complex practice changes such as implementing an evidence-based psychotherapy. Furthermore, the substantial increase in CBT usage among the facilitation group was achieved at a modest cost.</p

Evolving the theory and praxis of knowledge translation through social interaction: a social phenomenological study

Background: As an inherently human process fraught with subjectivity, dynamic interaction, and change, social interaction knowledge translation (KT) invites implementation scientists to explore what might be learned from adopting the academic tradition of social constructivism and an interpretive research approach. This paper presents phenomenological investigation of the second cycle of a participatory action KT intervention in the home care sector to answer the question: What is the nature of the process of implementing KT through social interaction? Methods: Social phenomenology was selected to capture how the social processes of the KT intervention were experienced, with the aim of representing these as typical socially-constituted patterns. Participants (n = 203), including service providers, case managers, administrators, and researchers organized into nine geographically-determined multi-disciplinary action groups, purposefully selected and audiotaped three meetings per group to capture their enactment of the KT process at early, middle, and end-of-cycle timeframes. Data, comprised of 36 hours of transcribed audiotapes augmented by researchers\u27 field notes, were analyzed using social phenomenology strategies and authenticated through member checking and peer review. Results: Four patterns of social interaction representing organization, team, and individual interests were identified: overcoming barriers and optimizing facilitators; integrating \u27science push\u27 and \u27demand pull\u27 approaches within the social interaction process; synthesizing the research evidence with tacit professional craft and experiential knowledge; and integrating knowledge creation, transfer, and uptake throughout everyday work. Achieved through relational transformative leadership constituted simultaneously by both structure and agency, in keeping with social phenomenology analysis approaches, these four patterns are represented holistically in a typical construction, specifically, a participatory action KT (PAKT) model. Conclusion: Study findings suggest the relevance of principles and foci from the field of process evaluation related to intervention implementation, further illuminating KT as a structuration process facilitated by evolving transformative leadership in an active and integrated context. The model provides guidance for proactively constructing a \u27fit\u27 between content, context, and facilitation in the translation of evidence informing professional craft knowledge

Organizational factors and depression management in community-based primary care settings

Abstract Background Evidence-based quality improvement models for depression have not been fully implemented in routine primary care settings. To date, few studies have examined the organizational factors associated with depression management in real-world primary care practice. To successfully implement quality improvement models for depression, there must be a better understanding of the relevant organizational structure and processes of the primary care setting. The objective of this study is to describe these organizational features of routine primary care practice, and the organization of depression care, using survey questions derived from an evidence-based framework. Methods We used this framework to implement a survey of 27 practices comprised of 49 unique offices within a large primary care practice network in western Pennsylvania. Survey questions addressed practice structure (e.g., human resources, leadership, information technology (IT) infrastructure, and external incentives) and process features (e.g., staff performance, degree of integrated depression care, and IT performance). Results The results of our survey demonstrated substantial variation across the practice network of organizational factors pertinent to implementation of evidence-based depression management. Notably, quality improvement capability and IT infrastructure were widespread, but specific application to depression care differed between practices, as did coordination and communication tasks surrounding depression treatment. Conclusions The primary care practices in the network that we surveyed are at differing stages in their organization and implementation of evidence-based depression management. Practical surveys such as this may serve to better direct implementation of these quality improvement strategies for depression by improving understanding of the organizational barriers and facilitators that exist within both practices and practice networks. In addition, survey information can inform efforts of individual primary care practices in customizing intervention strategies to improve depression management.http://deepblue.lib.umich.edu/bitstream/2027.42/78269/1/1748-5908-4-84.xmlhttp://deepblue.lib.umich.edu/bitstream/2027.42/78269/2/1748-5908-4-84-S1.PDFhttp://deepblue.lib.umich.edu/bitstream/2027.42/78269/3/1748-5908-4-84.pdfPeer Reviewe

EQUIP: Implementing chronic care principles and applying formative evaluation methods to improve care for schizophrenia: QUERI Series

<p>Abstract</p> <p>Background</p> <p>This paper presents a case study that demonstrates the evolution of a project entitled "Enhancing QUality-of-care In Psychosis" (EQUIP) that began approximately when the U.S. Department of Veterans Affairs' Quality Enhancement Research Initiative (QUERI), and implementation science were emerging. EQUIP developed methods and tools to implement chronic illness care principles in the treatment of schizophrenia, and evaluated this implementation using a small-scale controlled trial. The next iteration of the project, EQUIP-2, was further informed by implementation science and the use of QUERI tools.</p> <p>Methods</p> <p>This paper reports the background, development, results and implications of EQUIP, and also describes ongoing work in the second phase of the project (EQUIP-2). The EQUIP intervention uses implementation strategies and tools to increase the adoption and implementation of chronic illness care principles. In EQUIP-2, these strategies and tools are conceptually grounded in a stages-of-change model, and include clinical and delivery system interventions and adoption/implementation tools. Formative evaluation occurs in conjunction with the intervention, and includes developmental, progress-focused, implementation-focused, and interpretive evaluation.</p> <p>Results</p> <p>Evaluation of EQUIP provided an understanding of quality gaps <it>and </it>how to address related problems in schizophrenia. EQUIP showed that solutions to quality problems in schizophrenia differ by treatment domain and are exacerbated by a lack of awareness of evidence-based practices. EQUIP also showed that improving care requires creating resources for physicians to help them easily implement practice changes, plus intensive education as well as product champions who help physicians use these resources. Organizational changes, such as the addition of care managers and informatics systems, were shown to help physicians with identifying problems, making referrals, and monitoring follow-up. In EQUIP-2, which is currently in progress, these initial findings were used to develop a more comprehensive approach to implementing and evaluating the chronic illness care model.</p> <p>Discussion</p> <p>In QUERI, small-scale projects contribute to the development and enhancement of hands-on, action-oriented service-directed projects that are grounded in current implementation science. This project supports the concept that QUERI tools can be useful in implementing complex care models oriented toward evidence-based improvement of clinical care.</p

Integration of robotic surgery into routine practice and impacts on communication, collaboration, and decision making: A realist process evaluation protocol

Background: Robotic surgery offers many potential benefits for patients. While an increasing number of healthcare providers are purchasing surgical robots, there are reports that the technology is failing to be introduced into routine practice. Additionally, in robotic surgery, the surgeon is physically separated from the patient and the rest of the team, with the potential to negatively impact teamwork in the operating theatre. The aim of this study is to ascertain: how and under what circumstances robotic surgery is effectively introduced into routine practice; and how and under what circumstances robotic surgery impacts teamwork, communication and decision making, and subsequent patient outcomes. Methods and design: We will undertake a process evaluation alongside a randomised controlled trial comparing laparoscopic and robotic surgery for the curative treatment of rectal cancer. Realist evaluation provides an overall framework for the study. The study will be in three phases. In Phase I, grey literature will be reviewed to identify stakeholders' theories concerning how robotic surgery becomes embedded into surgical practice and its impacts. These theories will be refined and added to through interviews conducted across English hospitals that are using robotic surgery for rectal cancer resection with staff at different levels of the organisation, along with a review of documentation associated with the introduction of robotic surgery. In Phase II, a multi-site case study will be conducted across four English hospitals to test and refine the candidate theories. Data will be collected using multiple methods: the structured observation tool OTAS (Observational Teamwork Assessment for Surgery); video recordings of operations; ethnographic observation; and interviews. In Phase III, interviews will be conducted at the four case sites with staff representing a range of surgical disciplines, to assess the extent to which the results of Phase II are generalisable and to refine the resulting theories to reflect the experience of a broader range of surgical disciplines. The study will provide (i) guidance to healthcare organisations on factors likely to facilitate successful implementation and integration of robotic surgery, and (ii) guidance on how to ensure effective communication and teamwork when undertaking robotic surgery